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(57) ABSTRACT 

A multiprocessor system having a total number of available 
CPUs partitioned into one or more smaller pools of CPUs 
called servers where the number of CPUs available to a 
server is reduced below the total number of available CPUs. 
Software licensing costs are thereby reduced because the 
number of CPUs available to run the operating system or 
IS V software has been reduced to the number of CPUs in the 
pool of the server rather than the total number of available 
CPUs in the multiprocessor system. In order to enforce the 
isolation of CPUs required by software licensing, separate 
identification codes, CPUIDs, that contain unique system 
serial numbers are assigned to each server in the mulnpro- 
cessing system. The multiprocessor system has multiple 
CPUIDs, one for each server (each pool of CPUs that can 
execute operating systems and ISV software). 
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MULTIPROCESSOR SERVERS WITH system capacity and providing the new CPUID version 

CONTROLLED NUMBERED OF CPUS code when necessary. 

BACKGROUND OF THE INVENTION 4. Update ISV software encoded CPUID tables. 

The present invention relates to the field of computers and 5 - Take a system outage to implement the upgrade, 
more particularly to multiprocessor systems having CPUs 5 Steps 1-4 are normally performed days or even weeks 

that can be pooled for system operation. before the outage is taken. While tew software products 

Large-scale modern data processing systems have an actually check the CPUID version code for compliance, the 

architecture that embodies multiple CPUs that are closely customer is normally required by contract to notify both the 

integrated to form multiprocessor systems. The CPUs in Operating System vendor and any ISVs when their licensed 
such multiprocessor systems operate under control of an 10 software will be run on a different hardware configuration. 

Operating System. Large scale operating systems include The current practice often encounters delays that are waste- 

OS/390, VM/ESA, UTS, VSE/ESA and TPF. Some multi- ful to the computer user. 

processors are well-known to be have domains where each The above-identified cross-referenced application 

domain is assigned one instance of one operating system and describes a multiprocessor system having a plurality of 
is started with an IPL (initial program load) of that operating 15 CPUs that can be dynamically reconfigured between online 

system. More than one operating system, which can be and offline without system shutdown. The multiprocessor 

different operating systems or different instances of the same system is operable in different modes, including a user mode 

operating system, run in the multiple domains of the mul- for processing user programs and a system mode for pro- 

tiproccssor system, one operating system per domain. A cessing system programs unavailable to users. Although the 
feature of domains enables one or more logical processors 20 dynamic reconfiguration of that cross-referenced application 

(LPs) to be established for each domain. A logical processor makes moving CPUs between online and offline modes easy, 

is a logical entity that the operating system creates, controls the problem still remains of isolating ISV, OS or other 

and assigns to programs, frequently called independent programs from user control such that reductions in software 

software vendor (ISV) programs, such that each program fees below those charged for all CPUs in the full multipro- 
interprets the assigned LP as its own "virtual CPU". The 25 cesser configuration might be obtained, 

actual fiinctioning of the LP may be on a single one of the Accordingly, and in light of the problems of prior 

multiple CPUs in the multiprocessor system or may be systems, there is a need to be able to isolate programs to 

distributed across multiple ones of the CPUs on a time- certain selected ones of the CPUs in a multiprocessor system 

shared basis with other LPs. and to upgrade computer CPU configurations dy namically 
Computer users are concerned about the cost and avail- 30 without causing the computer user to suffer the delays and 

ability of their computer systems and particularly upgrades outages that have heretofore been required It is desirable 

and downgrades in the CPU configuration. A CPU upgrade mat users be able to purchase additional capacity quickly 

is performed, among other reasons, to increase the capacity and with little effort 
of the computer system and a CPU downgrade is performed 

to reduce the capacity of the system. Increasing or decreas- 35 SUMMARY OF THE INVENTION 
ing the capacity of a system, besides tailoring the system to The present invention is a multiprocessor system having 
fit the data processing needs of the user, also has a significant a phirality of cp Us tha t m configured into different server! 
cost element since many software products that run on The present invention partitions the total number of avail- 
computer systems are frequently priced! to the user as a ablc CPUs intoone or more smaller pools of CPUs c alled 
function of the system capacity and the CPU configuration. servers wJiei^ tgiu^beTof CPUs available to a server is 
Typically, software licensing costs are computed based on r educed below the total number of available C PUs. Software 
the number of CPUs that are available to run operating Bcenslng costs are thereby re duced becansTthe number of 
systems and Independent Software Vendors (ISVs) software CPUs available to run the operating system or ISV software 
in the full system. In general, the software licensing costs 4S has been reduced to the number of CPUs in the pool of the 
have been charged based upon the total number of available server rather than the total number of available CPUs in the 
CPUs in the full multiprocessor system since generally all multiprocessor system. 

such CPUs have been available to run the installed software In order to enforce the isolation of CPUs required by 

if called upon to do so under user election and control. software licensing, separate identification codes, CPUIDs, 

In large multiprocessor computer systems that employ 50 that contain unique system serial numbers are assigned to 

multiple system control programs (SCPs), CPU upgrades cac b serve r in the multiprocessing system . A CPUID typi- 

have been difficult to schedule because the computer user caTI^incTiiaS a veraoncode which identifies the number of 

must plan an outage for all of the SCPs running on the CPU CPUs allocate tZlf TlZZn ^ a unique system serial 

to be upgraded. With the continued exploitation of logical number for the server. The multiprocessor system has mul- 

partitioning using mul&ple domain features (MDF), it has 55 fpurwim. nn. for^ pool of CPUs that 

become impractical to schedule all SCPs to be down at the can execute o perating systems and ISV software), 

same time This difficulty causes customers to delay neces- ^ mu]tiproccssor system supports pools of CPUs of 

sary upgrades rather than attempting to schedule the fuU diffcrcnt typcTTclumng, for exaVnple, se^rs, spares and 

system outage. The current upgrade^rocess mchides the ^ mese of ^ QlAy ^ 

°, W / ng J. . , t . . , 6 0 servers a re assigned operating systems and ISV software on 

1. Establish a hardware upgrade price with the hardware w hich software licensing costs are computed. Generally, a 
. V ™ d J' 1 ~ . 0 CPUihat is assigned to run Coupling Control Code (CCQ 

2. Notify the Operating System Vendor (for example, is not in a server pool and can be excluded from licensing 
IBM for mam frame systems) to change their software fees. Similarly, spire CPUs are not members of any server 
charges to account for the upgraded system capacity. 65 pool and can be excluded from licensing fees. Spare CPUs 

3. Notify Independent Software Vendors (ISVs) to change are available to replace CPUs in any pool. The number of 
their software charges to account for the upgraded CPUs assigned, to the different pools is under feature control 
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where feature control is a facility for controlling features of FIG. 2 depicts a multiprocessor system having a new CPU 

the miUtiproccssor system The feature control facility is oot configuration in reconfigured servers 

accessible by end users of the multiprocessor system. CPU vir i a ,7. * . 

pool assignment for each domain isdone by the FMrt» ^ depicts a multiprocessor system having a new CPU 

Element Platform (PEP) and is not viewable from the < 00 ^^ tlon » ^configured servers. 

Hardware Management Console (HMC). The type of oper- 4 depicts a CPU configuration control subsystem 

ating system permitted to run in a CPU pool is also under within the multiprocessor system of FIG. 1, FIG. 2 and FIG. 

feature control. 3. 

The multiprocessor system can switch between online and FIG- 5 depicts further details of the I/O system within the 
offline without system shutdown. The multiprocessor system 1Q multiprocessor system of FIG. 1, FIG. 2 and FIG. 3. 
is operable in different modes, including a user mode for FIG. 6 depicts the logical processor (LP) configuration of 
processing user programs and a system mode for processing the FIG. 1 system 

system programs unavailable to users. Although the multi- FIG. 7 depicts the logical processor (LP) configuration of 
processor system is capable of being shutdown to terminate the FIG. 2 system. ^ 01 

operation and permit reconfiguration during or after a nn a j_ ' , . , „„ x 

shutdown, tbe multiprocessoTTystem of thepresent Invcl 15 J^^** 4 neW logIcal P rocessor M> 
tion includes a dynamic reconfiguration subsystem for ct^o j • 

reconfiguration without shutdown. 9 depicts a new logical processor (LP) upgrade 

The dynamic reconfiguration subsystem includes a scr- ^^f" ^ * ^ ^ 
vice processor having a feature file for identifying a current M ^ 10 depicts a new logical processor (LP) downgrade 
online number corresponding to a current number of online configuration. 

CPUs, a current offline number corresponding to a current ^9* depicts the relationship between servers, CPUs, 
number of offline CPUs and an update number correspond- domains and logical processors in the multiprocessor system 
ing to changes to be made in the current online number and of F * G * FIG * 2 ^ FIG. 3. 

the current offine number. A reconfiguration control unit is 25 FIG. 12 depicts tbe configuration for one server in the 
provided for reconfiguring CPUs in the multiprocessor sys- multiprocessor system of FIG. 1, FIG. 2 and FIG. 3 showing 
tern without being shutdown that includes a store for storing the CPUs and examples for the configuration of domains and 
configuration code in response to the feature file, a system logical processors in the multiprocessor system of FIG. 1, 
state execution unit for executing the configuration code to 2 FIG. 3. 

form configuration control information and decoder means « dftatt en nponjumnw 

for decoding the control information to change the current * Multinroc^r S^^r^ 
number of online CPUs and the current nurnber of offline r„l£^ Systems-FIG. 1 

lEUs by the update number . In FIG. 1, a multiprocessor system 1-0 is shown, inchid- 

The present invention allows the definition of offline ?! * *^? { ™* Mpe^y mduding CPUs 1-1, 
logical processors (LPs) which may be brougnTor^ wS 35 stioz ^r^^^T^'J^ ^J?**? * 
the rese^d^ap^ry is mn<\r BVRthhle. Thelotal number of r^.r^^^^ S ^ 
domainLPs, including offline LPs, may not exceedX^S S T mamtenan ^ of ^processor system 1-0. 

domain is activated, the number of online LPs per domain ^ DtPlatfonn ^ }ln ^ ch P™*^ ™ ^terface to 

may not exceed ^umfco^ „ ^^EE^ "* * hudmn 

-mmlina mntml rPfT E K uy 40 agement platform 1-22 for user hardware management. 

TTk!» a**t»i „ , Additionally, the multiprocessor system 1-0 includes an I/O 

I ^r>^ prDCeSS to return the system 1-23 for providing the CPUs 1 with I/O storage and 

T** ^ D ^ C ^^rT thC ^ ^ " is includes a storage system 1-26 for providing ^CTU^l 

retneved Wiethe Store CPUTD £TE>P iostruction. with main storage. The storage system 1 SSo2 soSwarS 

During SCP ^alizanon, a STTOP instruction is per- 45 executed by the CPUs 1 including, for example, operating 
formed tc ^retrieve : *e CFCTO and save it in storage. IBM system programs and user application programs. The op^ 
SCPs OS/390 L USC borage ^tion to establish ating system programs and user apptication programs 

a umque dynamic path array identifier, determine recovery (including ISV programs) are typically licensed to the mul- 
J3? m *!? °^ Proccssor tiprocessor system user and under terms that require license 

(SCLP) actions. B V software products (in particular those 50 fees based upon the number of CPUs online and available to 
products that check their encoded CPUID tables for execute the programs. 

comphance) issue their own STIDP when that ISV software In the FIG. 1 example, ten of the sixteen CPUs are online 
15 T 1 *^?™ specifically CPUs 1-1 M, 1-9, 1-10, 1-13 and 1-14. 

The CPUID contains each vendor's model number (for The other CPUs, namely, 1-7, 1-8, 1-11, 1-12, 1-15 and 1-16 
example, Amdahl 700) as weU as a version code which 55 are off-line. Only the online CPUs are capable of executing 
identifies a specific server (for example, x*58V785). In the the operating system programs and user application pro- 

c ase of the Amdahl MGS 700, the X'5' indicates that the grams. 

CPUs are running at full speed and the-XS' defines that eight" ~ Typical multiprocessor CPU fc>ards contain two CPUs so 
physical CPUs are installed and available for customer use. that the multiprocessor system can be physically populated 

The foregoing and other objects, features and advantages » with two, four, . . . , sixteen CPUs. However, a user 
of the invention will be apparent from the following detailed (customer) may only wish to initially purchase a machine 
description in conjunction with the drawings. that contains, for example, three or four useable CPUs and 

BRIEF DESCRIPTION OF THE DRAWINGS purchase more useable CPUs at a later time. Later on, the 

. , . customer win be able to dynamically upgrade the multipro- 

CPU itt^T^f^ 4- 65 ^system 1-0 to tbe maximum number of useable CPUs 
S^SSS^^ anaDgCd m ~™ ^ ^-pbysicanyinsUUedm^ 
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Dynamic CPU Reconfiguration is the ability to increase or 
decrease the number of CPUs a customer has purchased and 
are physically available for use without requiring a system 
reset. This dynamic reconfiguration capability increases the 
availability of the multiprocessor system. With this ability, 
customers arc able to purchase additional CPUs in a very 
short period of time without unwanted downtime. 

In FIG. 1, the CPUs 1-1 1-16 are additional 

organized into pools that form servers. Server 2-1 includes 
a pool of CPUs that includes CPUs 1-1, 1-2, 1-5 and 1-6. 
Server 2-2 includes CPUs 1-3 and 14. Server 2-3 includes 
CPUs 1-9, 1-10, 1-13 and 1-14. The offline CPUs 1-7, 1-8, 
1-11, 1-12, 1-15 and 1-16 are not in any pool. 
Reconfigured Multiprocessor System — FIG. 2 

In FIG. 2, the multiprocessor system 1-0 of FIG. 1 has 
been reconfigured to change the offline CPUs 1-7 and 1-8 of 
FIG. 1 to online in FIG. 2. The change of the configuration 
from FIG. 1 to the configuration of FIG. 2 occurs dynami- 
cally without necessity of shut-down of the multiprocessor 
system 1-0. Also, the server 2-2 of FIG. 1 has been recon- 
figured to include CPUs 1-7 and 1-8 as well as CPUs 1-3 and 
14 within the server pool. Reconfigured Multiprocessor 
System— FIG. 3 

In FIG. 3, the multiprocessor system 1-0 of FIG. 1 has 
been reconfigured to change the offline CPUs 1-U, 1-12, 

1- 15 and 1-16 of FIG. 2 to online in FIG. 3. The change of 
the configuration from FIG. 2 to the configuration of FIG. 3 
occurs dynamically without necessity of shut-down of the 
multiprocessor system 1-0. Also, the server 2-4 of FIG. 3 has 
been configured to include CPUs 1-11, 1-12, 1-15 and 1-16 
within a pool as server 24. 

CPU Reconfiguration Control Unit — FIG. 4 

In FIG. 4, the CPU configuration control unit 3-0 operates 
in response to a feature file in the service processor 1-20 for 
dynamically configuring the multiprocessor system 1-0. The 
feature file includes information identifying a current online 
number corresponding to the current number of online 
CPUs, a current online number corresponding to a current 
number of offline CPUs and an update number correspond- 
ing to changes to be made in the current online number and 
the current offline number. In the FIG. 1 example, the 
number of online CPUs is ten and the number of offline 
CPUs is six. In changing from the FIG. 1 configuration to the 
FIG. 2 configuration, the update number is two. The update 
number adds two to the online number and subtracts two 
from the offline number. 

While an increase in online CPUs and a decrease in 
off-line CPUs is contemplated going from the FIG. 1 con- 
figuration to the FIG. 2 configuration, the present invention 
works equally well in either direction. Namely, if the current 
configuration is that of FIG. 2 and the new configuration is 
that of FIG. 1, then the update number remains two, but the 
update number is subtracted from the online number and is 
added to the offline number. 

Also in FIG. 4, the feature file contains the server data and 
the CPU data that identifies the CPUs in each server 2-1, 2-2, 

2- 3 and 2-4. The feature file loading and control is not 
acccssable or useable by the end user of the multiprocessor 

^system 1-0 but-can onry be accessed by the vendor of the 
multiprocessor system 1-0 through customer representatives 
or other vendor access. Accordingly, the assignment of 
CPUs to pools to establish servers cannot be modified 
directly by the user of the multiprocessor system 1-0 and 
hence ISVs and others can rely upon the limitations in the 
size and capacity of servers in the multiprocessor system 1-0 
for purposes of licensing fees, performance allocation and 
control 
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Within FIG. 4, the user of the multiprocessor system 1-0 
is able to establish domains through control of the domain 
data, allocate logical processors (LPs) in the domains and 
allocate channels in the I/O system through I/O CDS data. 
5 Multiprocessor I/O Configuratiorj — FIG. 5 

In FIG. 5, the connection of the CPUs 1, including CPUs 
1-1, 1-2, .... 1-16 connect to the I/O system 1-23. The I/O 
system 1-23 an I/O CDS mapper 1-24 that determines the 
connections of each domain to particular ones of the 512 
channels 1-25. The I/O CDS mapper 1-24 maps the domains 
and channels 1-25 under control of the I/O CDS data from 
the HS A store 3-1 of FIG. 4. 
LP Configuration — FIG. 6. 

Each- of the 1 to 15 domains ca n have from 1 to 16 LPs. 
These IPs a re dispatched on CPUs 1-L . . . , 1-16 of the 
15 se rver to which the domain is assig ned in FIG. 1. FIG. 2 and 
FIG. 3. Dedicated LPs are dispatched 100% of the time on 
a dedicated CPU while shared LPs of each Qomain are 
di spatched a portion of the time on a sTSared CPtb Each 
domain typi cally can have separate system c ontrolprograms 
20 SCfe which each control operation in a w ell-known manner 
with a view o f the multiprocessor system which appears to 
be exclusive to the SCR Since each SCP runs independently, 
it is difficult to control all of the SCPs sud uhat tbey cm be 
shut down simultaneously in order to shut down the entire 
25 multiprocessor system 1-0. 

In FIG. 6, the state of each of the logical processors 
LP01, . . . , LP16 is either SHARED, DEDICATED or 
OFF-LINE. FIG. J^epresents the state of the logical pro- 
cessors of servers?' in FIG. 1, by way of example, where 
30 the dedicated logical processors are LP01_and LPQ2. The 
off-line logical processors in FIG. 1 are LP03, . . . , LP16 and 
no logical processors are shared. 

New Upgrade Configurations — FIG. 7, FIG. 8 and FIG. 9 
In FIG. 7, a new upgraded configuration of the domain 

35 logical processors executing on the CPU configuration of the 
FIG. 2 multiprocessor system is shown. In FIG. 7, the LP03 
and LP04 logical processors have been moved from the 
off-line state of the FIG. 6 configuration for the FIG. 1 
multiprocessor system to a shared configuration. The logical 

40 processors LP01 and LP02 remain dedicated. 

In FIG. 8, the logical processor LP03 is reconfigured 
relative to the FIG. 7 configuration to a DEDICATED status. 
The logical processors are LP01 and LPG2 remain dedicated 
and the logical processor LP04 remains shared. 

45 In FIG. 9, the logical processor LP05 is changed from the 
off-line status of FIG. 8 to a shared online status. The logical 
processors LP01, LP02 and LP03 remain dedicated and the 
logical processor LP04 remains shared. 
Downgrade Configuration — FIG. 10. 

50 In FIG. 10, the logical processor LP05 is moved from the 
online status as shown in the FIG. 9 configuration to an 
off-line status. LP03 is moved from dedicated to shared. The 
logical processors LP01 and LP02 remain dedicated and the 
logical processor LP04 remains shared. 

55 General Reconfiguration Operation 

The total number of domain LPs, including offline LPs, 
may not exceed the total number of physically installed 
CPUs. However, when a domain is activated, the number of 
^online LPs~ per-domain may- not exceed the number of 

60 customer purchased non-ICS CPUs. 

Once a dynamic upgrade is requested and approved, a 
new feature file is created and downloaded by an authorized 
technician of the multiprocessor vendor. This updated fea- 
ture file is then installed concurrently on the server to be 

65 upgraded. After the upgrade of the feature file is complete, 
the additional CPU capacity is immediately placed in the 
shared CPU pool for non-specific domain CPU requests. 
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Easting shared LPs in currently running domains will field, for each partition, the total number of LPs (offline and 

then have access to the extra CPU capaaty. Although their online) would have to equal the number of physically 

computed share wdl remain the same, the amount of shared installed CPUs. This relationship would create many offline 

CPU resource avajhible to each domain and its LPs has LPs that are not necessary and would create a big Jrobtem 

increased which will result in more effecuve system capacity 5 for the scheduler because the scheduler would be required to 

per xtomain. look at all of these LPs all of the time. Such operation would 

If the mimber of existing LPs restricts consumption of the also create a performance problem. Also, wirtTout the nun> 

new CPU resource, those predefined offline LPs may be ber of offline field, field domain operator can do a CF OT 

brought online via an SCP operator command such as an online for any of these offline LPs. This operation could also 

OS/390 U CF CPU(x),ONUNE" or VM/ESA "VARY ON 10 create performance problems. It is better to allow the system 

PROC(xr. The new CPUs will then be used to satisfy the operator to limit the number of offline LPs to the number that 

demand generated by these new online LPs. Additionally, night actually be used at upgrade time, 

the new CPU may be moved from the shared CPU pool to Wth addition of this new "number of offline proces- 

provide backing for a dedicated LP. If offline LPs were not fieU » modify their profiles after every upgrade or 

defined in the activation profile, a domain reactivation is - down-grade. Having this online processors field is a feature 

required. In this case, dynamic reorganization may be dis- 1 115615 ^ ^ wen if they aren't going to make use of 

ruptive to that domain. Once a dynamic upgrade is dynamic reconfiguratioa The profile contains a field for the 

completed, the activation profile should be updated to retain °^ clber of de ^ icated processors. The user can specify that all 

the new domain configuration. m dcdicatc<1 cvcn the offline ones. The scheduler 

If dynamic upgrade is used to dynamically increase ^ k * ted Processors in first-come first-serve order, 

capacity to handle a disaster, once the disaster is resolved the rS T Hr u J"* ^ online 6151 & l me Seated 

system may also be dynamically downgraded. The follow- irL **** SP*^** fewer dedicated than there are 

ing steps may be used to perform a dynamic downgrade: ^ mtotat 

1) Deactivate the backup domains or configure Lft offline rpn^T^^Tf f °' DCW 

as required. CPUs that may be added for an upgrade. Macrocode will set 

o\ nvnom^iiu h«™™^ # , _ , . „. 25 up the number of control blocks for a partition at creation 

fea^rT^ doWDgrade b * mst ^ a Dew time based on the number of logical processors specified iS 

« x* j*t !i l » , „. * e create partition Request/Response Control Block 

3) M«fafy and save changed domau, profiles. (RRCB). Tbe create partition RRCBb changed for dynamo 

Software asset management requires that the CPUID be reconfiguration so that the total number of togical processors 

aerate. When a Dynamic CPUUpgrade is performed, tbe 30 and the number of offline logical prcJssorc^Tb* 

CPUID, including tbe version code, is immediately updated included. At activation time, Macrocode will only bring 

t % al Z l 1lT™ i ?t C,mtat mo ^ l :^ atil *• * online the total number of logical processors minus thf 

re-IPLed, there will be a mismatch between the CPUID number of offline logical processors. 

b S 5f |S *** initiali2ation "* «•» «™ 3- The field engineer puts tbe machine into Concurrent 

returned by a SJIUF. 3S Maintenance (CM) mode so that dynamic installation of the 

Even with dynamic CPU reconfiguration, the user still has features tape can be done 

the responsibility of notifying aDsofrware vendors that an 4. From S VP, the field engineer installs a new features 

£f «t J}^ ^ 0peratan8 System vendor fpe from tbe Generation Frame (GE) using thelew 

should be advu»r that a dynamic upgrade wQl be performed dynamic reconfiguration option. This new futures tape 

so that they wdl better understand the environment in case 40 specifies the number of CPUs that are purchased whether iS 

^ff^^l r^"^ • more or less than before. The SVP decodes the new fear^ 

Detafled Dynamic CPU Reconfiguration Algorithm tape ,nd checks it. If there are some diflere«Js7bo^ 

The following steps present tbe algorithm for dynamic beyond the change in mimber of CPUs, then it is rejected 

recor^guratton.^ Also, if there is no change in the number of CPUs, a warnine 

1 The nraltiprocessor user (customer) may specify offline 4S will be issued but the user can continue if desired 

^^^Ti!T g , ^andactrvatmgdomains. Trx total number 5. To ensure that the customer has stopped use of the 

° £ ^ f Tl Dnmbcr ? f physically CPUs that are no longer purchased whcne^Tfeaturcs are 

installed CPUs The number of LPs may be less than the being down-graded, the SVP will issue an High Level 

total number of physicaUy installed CPUs. Tins operation Command (HLQ to tbe PEP querying whether there will be 

ST* C ^frl° ftf L S ? nUt l 0n ^ ement 50 eno "gh CPUs left after the down^de to prevent rmmi£ 

Platform (PEP). Anew field is added to the profiles to hold degraded based on tbe current LP needs. The PEP wfllin 
fce number of processors that are to be offline at activation turn need to query Macrocode. Note, that this needs loLz 
dme. The number of processors field holds the number of two HLC handshake. Tbe first HLC queries the Macrocode 
offlme plus online processors. The mimber of processors is The response to this HLC is sent back immediate^ fromThe 
checked to ensure : that it does not exceed the number of ss PEP. Then, the PEP queries macrocode for the information 
physKaUy mstaUed CPUs. A message is issued if the user Then, the PEP sends the information back in a second HLC 
exceeds this number. A field is addedto the processor This sequence is necessary in order to prevent timing out on 
cha^te^ profile edit, frame to.sbow_the number of the first HLC while waiting for Macrocode to respond. If the 
purchased CPUi The affect partition controls frame shows FE chooses to continue with a downgrade!^ beL? 
the number of LPs actually being used for active partitions 60 warned about a mismatch in tbe number of LPs and CPUs 
and the number m the profile for inactive partitions. men the machine may run degraded. For performance* 

A new field in tbe profiles holds the number of offline reasons, the customer should reduce the maximum number 
processors. Due to an incompatibility between tbe PEP and of shared domain LPs to the mimber of shared physical 
the Hardware Management Console (HMQ, in order to processors. Since multiple shared LPs -^»o run on fewer 
change theprofiles, a user uses Distributed Console Access 65 shared CPUs, this is not mandatory, but it is suggested 
I'^^to&toteW^^ofusiug 6. From the SVP, the FE acYvaE iie nTSres 



... - ,t . , , o «. kuui UK >j»r, uic re activates me new features 

the HMC screens directly. Without tbe number of offline without having to do an Initial Microcode Program Load 
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(IMPL) of tbe system. Tbe SVP updates the features control 
data on the hard drive and in memory and sets new CPUIDs 
into the HSA table. CPUFIRM uses toe new CPUID from 
the next Store CPUID. A detailed log containing machine 
configuration information is retained. Service Took need 
this information just like they do at IMPL time. 

7. The dynamic reconfiguraiton feature activation causes 
a new HLC to be broadcast to the PEPs specifying that the 
number of purchased CPUs has changed. 

8. The PEP syncs itself with tbe SVP. It updates all of it's 
information on how many CPUs are installed and purchased. 
It logs a message in the event bg to track tbe fact that the 
number of purchased CPUs has changed. 

9. The PEP sends Macrocode a new RRCB telling Mac- 
rocode that tbe number of purchased CPUs has changed. Tbe 
PEP also sends this RRCB to Macrocode whenever the SVP 
is IPLed just in case an IPL has caused this message to be 
lost. 

10. Macrocode issues a diagnose command to the SVP to 
obtain the maximum number of CPUs allowed by feature 
control data. Macrocode also issues this diagnose whenever 
the primary PEP has switched This will cover the case 
where the message to Macrocode may have been lost due to 
a primary PEP switch. Macrocode retains this data in a 
secure manner. 

11. Macrocode will only schedule CPUs that are within 
the maximum number allowed by feature control data. The 
'extra* CPUs are put in 'spare' state so that they can be used 
for dynamic CPU reconfiguration (DCR). 

12. "typically, all of the physically installed CPUs are reset 
at reset power on of the system so there is nothing needed 
to get any newly purchased CPUs reset. 

13. Macrocode schedules work on the new CPUs if there 
are LPs already available to use them in the case of an 
upgrade. In tbe case of a down-grade, Macrocode stops use 
of the CPUs that arc no longer purchased. This operation 
may cause the machine to run degraded if there aren't 
enough CPUs for all of the LPs that need them. The 
Macrocode scheduler will do LP-squeeze. Also, a degrada- 
tion machine check will be generated which MVS logs. 

14. Macrocode issues an RRCB to all PEPs indicating that 
a DCU/DCR operation took place. Ibis RRCB has the new 
mapping table for Extended Virtual Machine (EVM) to 
System CPU addresses. 

15. Macrocode sends RRCB(s) to all of the PEPs to 
indicate that the operator MAY vary the new CPU online to 
a partition if it is an upgrade and the partition has offline LPs 
that could be brought online. Since LPs can float across 
shared CPUs, there may be no need to bring any more LPs 
online. The PEP will receive as many RRCBs as there are 
partitions with LPs to be brought online. Tbe PEP will issue 
a pop-up messages to the user and issue log messages. 

16. Macrocode sends an RRCB to all of the PEPs indi- 
cating that the machine is running degraded if this has 
happened due to the down-grade. The PEP will issue a 
pop-up message to the user and log a message. There needs 
to be an indication in the RRCB as to whether this is serious 
and involves dedicated processors that aren't getting a full 
CPU (for example, 6 dedicated LPs from the same partition 
sharing 4 CPUs) or not so serious (for example, 6 LPs from 
the same partition sharing 4 CPUs without any being 
dedicated). 

17. For an upgrade, if so prompted and it is necessary, an 
operator issues a CF CPU (x) online from MVS, causing an 
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enough CPUs to go around, the LPs will be made dedicated. 
The SCLP CF CPU may fail if there aren't enough CPUs to 
go around. 

18. The customer may wish to change the number of 
online processors in the profiles so that the desired number 
are brought online when the partition is re-activated If the 
user doesn't change the profiles, the next partition activation 
may fail. 

19. Tbe customer may wish to change the number of 
dedicated CPUs for a given partition using the affect parti- 
tion controls frame. It should be noted that the CPUID is 
only stored in tbe LPB at domain activation and should 
always reflect the current version of the system. The LPB 
CPUID should not change for an already activated domain. 

EXAMPLES 

Based on a multiprocessor system that has six physically 
installed CPUs. 

Example 1 

1. User specifics 1 partition with 6 processors, 2 offline, 4 
dedicated in the profile. Purchased-4. 

Editing the profile causes No warning message. 

Partition activation causes a create partition RRCB to be 
sent to Macrocode. 6 processors, 2 offline, 4 dedicated are 
specified. 

Macrocode receives the RRCB. Macrocode will put 4 LPs 
online and 2 offline. 

The user may upgrade the machine to 6 CPUs. 

Upon doing so, still only 4 CPUs are being used until a CF 
CPU OK command is given from MVS. This causes an 
SCLP CF CPU to be sent to Macrocode. At that time, the 
newly on-lined LP's start to be scheduled. 

The user can do an affect partition control to change the 
number of dedicated CPUs if desired. 

If the user reactivates the partition, 4 LPs will be put 
online and 2 online. The user can and should edit the profile 
so that 6 LPs will be brought online automatically. 

The user can now down-grade the machine back to 4 
CPUs. 

If that is done, and no LPs were taken offline using tbe CF 
CPU OFF command from MVS, then a warning will be 
issued at feature installation time, warning the user that the 
machine will run degraded if he/she continues. 

Macrocode will put two of the CPUs to sleep. 

The Macrocode scheduler will do LP-squeeze. Also, a 
degradation machine check will be generated which MVS 
probably just logs. 

Example 2 

2. User specifies 1 partition with 4 processors, 0 offline, 4 
dedicated in tbe profile, purchased-4. 

Editing tbe profile to say 4 processors causes NO warning 
message. 

Partition activation causes a create partition RRCB to be 
sent to Macrocode. 4 processors, 4 dedicated, 0 online are 
specified. 

Macrocode receives the RRCB. You'll end up with 4 
online LPs, 0 offline. 
The user may upgrade the machine to 6 CPUs. 
Upon doing so, still only 4 CPUs will be used. The user 
hasn't specified extra LPs to run on the new CPUs so he 
can't make use of the new CPU power. 

If the user reactivates the partition, only 4 LPs will be put 



SCLP CF CPU to be sent to Macrocode. Macrocode starts 65 online since this is the number in tbe profile, 
scheduling more LPs oo the physical CPUs. If the user has The user could activate another partition to make use of 
specified that the new LPs should be dedicated and there are the extra CPUs. 
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Example 3 

3. The user specifies 1 partition, 6 processors, 2 offline, 0 
dedicated in the profile. Purchased -4. 

Partition activation causes a create partition RRCB to be 
sent to Macrocode. 6 processors, 2 online, 0 dedicated are 
specified. 

Macrocode receives ibe RRCB. Macrocode will put 4 LPs 
online and 2 offline. 

The user may upgrade the machine to 6 CPUs. 

Upon doing so, still only 4 CPUs are being used (an LP 
can use at most 100% of a CPU) until a CF CPU ON 
command is given from MVS. This causes an SCLP CF 
CPU to be sent to Macrocode. At that time, the newly 
on-lined LPs start to be scheduled. 

The user can do an affect partition control to change the 
number of dedicated CPUs if desired 

If the user reactivates the partition, 4 LPs will be put 
online unless the user edits the profile to say that their are 0 
online. 

The user can now down-grade the machine back to 4 
CPUs. 

If that is done, and no LPs were taken offline using the CF 
CPU OFF command from MVS, then a warning will be 
issued. This warning win be a gentle warning because there 
are enough CPUs for the LPs to share although each LP will 
be getting less than 100% of a CPU. 

Macrocode will put two of the CPUs to sleep. 

The 6 LPs will be online sharing 4 CPUs. 

Example 4 

4. The user specifies 2 partitions, purchased-4. 
Part 1: 4 processors, 0 offline, 0 dedicated in the profile Part 
2:4 processors, 2 offline, 2 dedicated in the profile Partition 
activation for Part 1 causes a create partition RRCB to be 
sent to Macrocode. 4 processors, 0 online, 0 dedicated are 
specified. 

Macrocode receives the RRCB. Macrocode will put 4 LPs 

o nlin e. 

Partition activation for Part 2 causes a create partition 
RRCB to be sent to Macrocode. 4 processors, 2 online, 2 
dedicated are specified. 

Macrocode receives the RRCB. Macrocode will put 2 
more LPs online and 2 LPs online. There are a total of 6 LPs 
online and 2 LPs offline now and there are 4 CPUs available. 
Two of the LPs have dedicated CPUs and the rest are sharing 
the other two CPUs. 

The user may upgrade the machine to 6 CPUs. 

Upon doing so, all 6 CPUs are being used. A CF CPU ON 
command is not necessary since there are enough LPs 
already online to make use of the new CPUs. However the 
user could do a CF CPU ON for the two offline LPs if 
desired. It will just add to the number of LPs sharing the 
CPUs. 

The user can do an affect partition control to change the 
number of dedicated CPUs if desired. 

If the user reactivates the partitions, 6 LPs will be put 
online and 2 LPs will be offline. The user should edit the 
profile if he/she doesn't want any offline LPs. 
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Server Configurations — FIG. U and FIG. 12 

In FIG. LL, further details of the configuration of servers 
2 injbc multiprocesso r systems ^0 of the present invention 
are shown. In FIG. 11, the servers 2 include servers 1, . . . , 
S where in the 1ft rPTi ^inhnrii^^ Ascribed, S is a 
maximum of IS. Each of the servers 2 includes one or more 

CP UTl including CP Us 1 C . where for server 1, 

Cj-Cj, and so on whereTbr server S, C,-Q. Each of the 
values C lf . . . , C„ . . . , C s is some value from I to 15 for 

t he 16 CPU ^ mhodiWji t described. " 

laHfljauD ^Eamplc of the FIG. 11 j urther details of the" 
configuration of servers 2 m the multiprocessor systems 1-0 
of FIG. 11 is shown fortEe server 2-1 of FIG. 1. In FIG. 12, 
thr y rwr ?rl inrl ndes CPUs L . . . . C w here for server 2- 1 . 
C t -4 for the. CPl 1 -l^l -XT-5 a nd 1-6. Further in the FIG. 
12 example, the server has been configured with two / 
domains, DOMAIN 1 and DOMAIN 2. Still further,! 
DOMAIN 1 has been ™ nt1 g ,1 r p tfi wj th si x Lfo, nam ely \ 
LPl^. - . , LP6 and DOMAIN 2 has been configured with ' 

seyerTLPs, namely LP1 LP7. ^ 

20 Multiple Server Facility (MSF) System Level Specification 
Since software licensing costs are computed based on the 
number of CPUs that are available to run operating systems 
and ISV software, the present invention partitions the total 
number of available CPUs into one or more smaller pools of 
25 CPUs called servers where the number of CPUs available to 
a server is reduced be tow the total number of available 
CPUs. Software licensing costs are thereby reduced because 
the number of CPUs available to run the operating system or 
ISV software has been reduced to the number of CPUs in the 
30 pool rather than the total number of available CPUs in the 
multiprocessor system, 

MSF supports pools of CPUs for different types of opera- 
tions. The different types of operations include, for example, 
servers, spares and Coupling Control Code (CCQ. Of these 
35 types of pools, the servers arc the ones to which operating 
systems and ISV software arc assignable and which are, 
therefore, the ones upon which software licensing costs arc 
computed. Generally, a CPU that is assigned to run Coupling 
Control Code (CCC) is not in a server pool and can be 
excluded from licensing fees. Similarly, spare CPUs are not 
members of any server pool and and can be excluded from 
licensing fees. Spare CPUs are available to replace CPUs in 
any pooL The number of CPUs assigned to the different 
pools is under feature control where feature control is a 
45 facility for controlling features of the multiprocessor system. 
The feature control facility is not accessible by end users of 
the multiprocessor system. CPU pool assignment for each 
domain is done by the Presentation Element Platform (PEP) 
and is not be viewable from the Hardware Management 
Console (HMC). The type of operating system permitted to 
run in a CPU pool is also under feature control. 

In order to enforce the isolation of CPUs required by 
software licensing, separate identification codes, CPUIDs, 
that contain unique system serial numbers are assigned to 
each server in the multiprocessing system. For example, a 
CPUID typically includes a version code which identifies 
thejaumbcx of CPUs allocated-to the server and a unique 



40 



50 



The user can now down-grade the-machinc-back-to 4 system -serial' number- for the servcrr The multiprocessor 

M T " system has multiple CPUIDs, one for each server (each pool 

60 of CPUs that can execute operating systems and ISV 
software). 

The feature control file specifies the number of CCC 
CPUs, the number of server CPUs and the number of spare 
CPUs which numbers sum to the total number of CPUs in 
the multiprocessor system. There is one size field per server 
pool where the sum of the size fields for all servers equals 
to N (the number of purchased processors). 



CPUs. 

If that is done, and no LPs were taken online using the CF 
CPU OFF command from MVS, then a war ning will be 
issued. This warning will be a gentle warning because there 
are enough CPUs for the LPs to share although each LP will 
be getting less than 100% of a CPU. 

Macrocode will put two of the CPUs to sleep. 

The 6 LPs will be online. 4 of the LPs will be sharing 2 
CPUs and 2 LPs will have dedicated CPUs. 



65 
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Dynamic CPU upgrade and downgrade may be done 
between CPU pools with the feature file. The CPUID of each 
server formed by a CPU pool is defined by the feature flic. 
Individual CPUs or CPUs in a server may be degraded. The 
speeds (degrade values) of CPUs may nave different values . 5 
in different CPU servers. The speeds (degrade values) of 
CPUs within a particular server are identical. A spare CPU, 
if present in the multiprocessor system configuration, may 
be used to replace any CPU in any server. The spare CPU 
must be set to the same speed as the CPU it has replaced io Q 
the CPU server. 

Although a multiprocessor end user may run CCC in a 
server pool rather than 00 a dedicated CPU dedicated to 
CCC, such operation subjects the running of CCC to stan- 
dard server operating system licensing fees. 15 

There are one or more active logical Input/Output Con- 
figuration Data Sets (IOCDSs) in the multiprocessor system 
for controlling the . mapping of I/O channels among the 
domains. 

NodelD — The System Hardware Number NodelD is M 
selected by the system (the first one defined). 

Channel Subsystem Call (CHSQ, Service Call Logical 
Processor (SCLP), ESCON Multiple Interface Facility 
(EMIF) & Dynamic Reconfiguration Memory (DRM) 
operations are not restricted to those facilities available to a ^ 
server. 

Diagnose 204 and Request Machine Information (RMI) 
information may be provided for the domain, pool or entire 
system as specified by the customer in the domain profile. 
The information returned by the Diagnose 204 interface is ^ 
under operator control such that it provides the following 
scope of reporting options: 

a) Information for all CPUs in the system. 

b) Information for all CPUs in a general pool and the 
coupling facility pool. 3S 

c) Information for only those CPUs in the pool in which 
the domain is allocated. 

d) Information for only the domain that issued the diag- 
nose 204. 

ISV Software Checking of CPUID 40 

To check the CPUID for licensing compliance the ISV 
software periodically does a STIDP (Store CPU ID). The 
STIDP instruction returns the CPUID of the MSF server on 
which it is installed. The ISV program then accesses a table 
to obtain a coded table entry provided by the ISV as part of 45 
the ISV software purchase and installation process. The 
encoded table entry is decoded by the ISV software and 
compared with the CPUID to check for compliance. By 
checking for compliance the ISV ensures that the customer 
is running its software only on servers licensed to run that 50 
ISV software. If not in compliance, the ISV software logs 
the non-compliance and takes any other action that the ISV 
software vendor has embedded in the software. For example, 
a warning of tbe license violation is displayed and/or ulti- 
mately the ISV software is disabled or otherwise restricted 55 
from full performance until the license violation is corrected. 
Multiple Server Facility Algorithm 
— 1 .-The -features-file specifies a count, operating system 
~mode7 and "system serial number for each of servers. The 
sum of the counts equals N where N is the total number of 60 
authorized CPUs in tbe multiprocessor system. Changes in 
CPU counts can be activated dynamically. 

2. Changes in the operating system mode and system 
serial number of an existing server are activated statically. 
New CPU servers may be added dynamically. There need 65 
only be one model number on the features file. The model 
number does not have any indication of CPU number or of 



whether the machine includes a coupling facility (CCC) or 
multiple servers since these determinations in tbe feature file 
are separately determined. 

3. At dynamic features file installation, a check is made to 
ensure that the number of CPUs takes into account all CPUs 
that are in the full multiprocessor system. 

4. A Jynamic features installation that is a down-grade in 
the number of CPUs checks to see if the server will enter a 
degraded mode once the features are activated. The check 
consists of an High Level Command (HLC) to the PEP, a 
response to the HLC, a Request/Response Control Block 
(RRCB) from the PEP to Macrocode, a response to the 
RRCB with the information from Microcode, and then an 
HLC from the PEP to the Service Processor (SVP) with the 
information. The information passed back accounts for all 
servers and CPUs in the multiprocessor system. There can 
be degradation for the CCC CPUs and not the servers or vice 
versa. Or, there can be degradation in both or neither. 

5. There can be a reduction in the number of servers but 
only if there are no active domains assigned to the CPUs in 
the servers being removed. When the SVP queries the 
degradation status it will be provided a degradation warning 
by the PEP if there win be no CPUs in a pool with active 
domains. If the user continues after receiving a degradation 
warning or if there was no degradation warning, then the 
SVP updates its features control data on the hard drive and 
in memory. It stores a new CPUID into HSA. CPUFIRM 
uses the new CPUID for the next store CPUID. Hard drive 
feature control data holds CPU counts for all servers, the 
operating system mode and system serial number associated 
with each server. 

6. An HLC is sent to the PEPs notifying them of the 
change in CPU number. 

7. The PEP updates its information and sends an RRCB to 
Macrocode with notification of the change in CPU number. 

8. Macrocode issues a new diagnose command to the SVP 
to obtain the total number of CPUs purchased, the number 
of CPUs, operating system mode, unique system serial 
number, model number^nd performance level for server. 
The diagnose returns CPU counts for all servers. 

9. The Macrocode scheduler keeps track of tbe server's 
operating system mode in order to know on which CPUs the 
partition can run. There are two types of partitions. Tbe first 
is an CCC partition. This type of partition can run in a CPU 
pool with an operating system mode of CFCC or GEN- 
ERAL. The second is a non-CCC partition. This type of 
partition can only run in a CPU pool with a non-CCC 
operating system mode. Macrocode scheduler will have to 
account for all servers. 

10. The create partition RRCB accepts a server identifier 
(in tbe range of 0-F for a 16 CPU system). 

11. At domain activation, Macrocode checks to see if 
there are enough of the right kind of CPUs with the correct 
operating system mode to allow the domain to be activated. 

12. The CPUID is stored in tbe Logical Processor Block 
(LPB) at domain activation. The LPB CPUID does not 
change for an already activated domain, unless an upgrade 
or-downgrade is-performcd.- Macrocode creates different 
CPUIDs for different partitions depending on the server the 
partition resides in at domain activation. The version code 
field contains the count of CPUs in the server's CPU pool. 
The model number is the same for all partitions. The CPU 
Identification Number contains a unique system serial num- 
ber to support CPU pool separation. 

13. Macrocode schedules work on the new CPUs if there 
are LPs already available to use them if this is an upgrade. 
In the case of a down-grade, Macrocode stops use of the 



03/17/2004, EAST Version: 1.4.1 



IS 



US 6,453344 Bl 



CPUs that are no longer purchased. This stoppage may cause 
the pool to run degraded if there aren't enough CPUs for all 
of the LPs that need them. The Macrocode scheduler does an 
LP-squeeze. 

14. Macrocode issues an RRCB to all PEPs indicating that 
a DCU/DCD operation took place. This RRCB has the new 
mapping table for Extended Virtual Machine ^EVM) to 
System CPU addresses. 

15. Macrocode sends RRCB(s) to all of the PEPs to 
indicate that the operator may vary the new CPU online to 
a partition if it is an upgrade and the partition has offline LPs 
that could be brought online. Since LPs can float across 
shared CPUs, there may be no need to bring any more LPs 
online. The PEP will receive as many RRCBs as there are 
partitions with LPs to be brought online. The PEP will issue 
a pop-up message to the user and issue log messages. 

16. Macrocode sends an RRCB to all of the PEPs indi- 
cating that the pool is running degraded if this has happened 
due to the down-grade. The PEP will issue a pop-up message 
to the user and log a message. 

17. For an upgrade, if so prompted and it is necessary, an 
operator issues a CF CPU (x) online from MVS, causing an 
SCLP CF CPU to be sent to Macrocode. Macrocode starts 
scheduling more LPs on the physical CPUs. If the user has 
specified that the new LPs should be dedicated and there are 
enough CPUs to go around, the LPs will be made dedicated. 
The SCLP CF CPU may fail if there aren't enough CPUs to 
go around. 

18. The customer may wish to change the number of 
offline processors and the CPU pool in the profiles so that the 
desired number of CPUs are brought online and the right 
pool of CPUs are used when the partition is re-activated. If 
the user doesn't change the profiles, the next partition 
activation may fail. 

While the invention has been particularly shown and 
described with reference to preferred embodiments thereof; 
it will be understood by those skilled in the art that the 
foregoing and other changes in form and details may be 
made therein without departing from the scope of the 
invention. 

What is claimed is: 

1. A multiprocessor system having a plurality of CPUs 
operable in different modes, including a user mode for 
processing user programs and a system mode for processing 
system programs unavailable to users, said multiprocessor 
system having a dynamic configuration subsystem for opera- 
tion in the system mode comprising: 

a service processor having a feature file for identifying a 
current online number of CPUs and for identifying one 
or more server numbers of CPUs where each server 
number is less than said online number, 

a configuration control unit for configuring CPUs in the 
multiprocessor system, 

store means for storing code in response to the feature 
file including storing said online number and said 
server numbers, 

system state execution means for executing the code to 
form configuration control information, 
. - decoder-means fordecoding the control information to 
partition said CPUs into servers, each server having 
a number of CPUs equal to a corresponding one of 
said server numbers. 

2. The system of claim 1 wherein a sum of said server 
numbers is less than or equal to said online number. 

3. The system of claim 1 including means for assigning 
domains to each of said servers. 

4. The system of claim 3 including means for assigning 
logical processors to each of said domains. 
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5. The system of claim 1 including means for assigning a 
unique identification number to each of said servers. 

6. The system of claim 5 wherein each identification 
number includes an indication of the server number for 

5 identifying how many CPUs are in the server and a unique 
serial number for the server. 

7. The system of claim 6 wherein said multiprocessor 
system executes software limited for use on pools of CPUs 
not exceeding an authorized number of CPUs and wherein 

10 said software is enabled to run on one of said servers havin g 
a server number not exceeding said authorized number and 
is not enabled to run on any server having a server number 
exceeding said authorized number. 

8. The system of claim 7 wherein said software includes 
15 means for checking the server number of servers enabled to 

run the software to determine if the server number is less 
than or equal to the authorized number, 

9. The system of claim 5 wherein the identification 
number is CPUID and wherein the multiprocessor system 

2Q includes one or more CPUIDs, one for each server. 

10. The system of claim 1 wherein said plurality of CPUs 
includes offline CPUs. 

U. The system of claim 1 capable of being shutdown to 
terminate operation, and wherein, 
25 said service processor identifies in said feature file a 
current offline number corresponding to a current num- 
ber of offline CPUs and an update number correspond- 
ing to changes to be made in the current online number 
and the current offine number, 
30 said configuration control unit reconfigures CPUs in the 
multiprocessor system without being shutdown through 
operation of said decoder means decoding the control 
information to change the current number of online 
CPUs and the current number of offline CPUs by the 
35 update number. 

12. The system of claim U wherein the reconfiguration is 
an upgrade to add to the current number of online CPUs. 

13. The system of claim U wherein the reconfiguration is 
an downgrade to subtract from the current number of online 

40 CPUs. 

14. The system of claim 1 wherein said service processor 
identifies in said feature file different pools of CPUs includ- 
ing said servers. 

15. The system of claim 14 wherein said different pools of 
45 CPUs include coupling control code CPUs. 

16. The system of claim 14 wherein said different pools of 
CPUs include spare CPUs. 

17. The system of claim 1 wherein said service processor 
identifies in said feature file different pools of CPUs includ- 

50 ing said servers, including coupling control CPUs and 
including spare CPUs. 

18. A multiprocessor system having a plurality of CPUs 
operable in different modes, including a user mode for 
processing user programs limited for use on pools of CPUs 

55 not exceeding an authorized number of CPUs and a system 
mode for processing system programs unavailable to users, 
said mul tiprocessor system having jt dynamic configuration 
— subsystem for operation in-the system mode comprising: 

a service processor having a feature file for identifying a 
60 current online number of CPUs and for identifying one 
or more server numbers of CPUs where each server 
number is less than said online number, 
means for assigning a unique identification number to 
each of said servers, each identification number includ- 
es ing an indication of the server number for identifying 
how many CPUs can used by the server for execution 
of said user programs. 
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a configuration control unit for configuring CPUs in the 

multiprocessor system, 
store means for storing code in response to the feature file 

including storing said online number and said server 

numbers, 

system state execution means for executing the code to 

form configuration control information, 
decoder means for decoding the control information to 

partition said CPUs into servers, each server having a 
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number of CPUs equal to a corresponding one of said 
server numbers, 
means to enable said user programs to run on one of said 
servers having a server number not exceeding said 
authorized number and means to inhibit said user 
programs to run on any server having a server number 
exceeding said authorized number. 

***** 
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